1. Field of the Invention
This invention relates generally to robotic arm positioning using an optical camera, and more specifically to a method of correcting for parallax errors inherent in positioning a robotic arm using a single off-axis optical camera.
2. Related Art
With the advent of robotics technology, automated processing has come into widespread use in numerous facets of today's industry. Robotic systems are employed to perform a myriad of functions from assembly line processes to materials processing to real-time operational processes. These robotic systems are often implemented using a computer-controlled robotic arm.
Applications for robotic arms in assembly line processes include welding, painting, and testing. In the materials processing environment, robotic applications can include, for example, locating materials in a processing environment such as a furnace or a chemical bath, and mixing or processing materials or chemicals. In real-time operational environments, robots are used to perform operational functions such as automated order picking and for computer operations, tape selection and mounting. An example implementation of the tape selection and mounting application is Nearline Automated Cartridge System (ACS), Model 4400, manufactured by Storage Technology Corporation.
To optimize performance of robotic systems in the above-mentioned and other applications, a robotic arm must be positioned quickly and precisely to perform its task. To illustrate this concept, the tape selection and mounting robotic system will be used as an example. In this example, the tape selection robotic system must locate a correct tape to be loaded, and quickly and precisely align its arm to select the proper tape. If the alignment is imprecise, a critical error may result (e.g., the robotic arm could miss the tape entirely or could retrieve the wrong tape). In addition, if the arm is extended when aligned imprecisely, damage to the tape, the robotic arm, or a tape storage bin may result.
Generally, a trade-off exists between the speed and precision with which a robotic arm may be aligned. In conventional systems, attaining a higher degree of alignment precision requires a greater alignment time. In addition if alignment is imprecise, retrieval must be done more slowly to minimize the amount of damage that could be caused by "crashing" the misaligned arm into a bin or a tape cartridge.
Further, a higher degree of precision means that the systems can be designed to tighter specifications. For the tape selection example, this means that bins which house the tape cartridges can be made smaller and positioned more closely to one another. As a result, system size is reduced and tape access time is quicker because the robotic arm has less distance to travel between tapes.
Many conventional systems employ a camera as a means for positioning the robotic arms. In these systems, the camera is mounted to the robotic arm and moves with the robotic arm. The camera, in effect, becomes the `eyes` of the robotic system. A controller within the robotic system uses the camera to search for a known pattern, called a target. The controller receives electronic signals from the camera indicating the location of the robotic arm with respect to the target. The controller then aligns the robotic arm using that target as a positioning guide.
The camera typically uses a solid-state image sensor array such as a Charge-Coupled Device (CCD) array to sense the target. The sensor array comprises a matrix of discrete photosensing elements. Each sensing element of the solid-state image sensor array is referred to as a picture element, or pixel. Each photosensing element generates a charge packet which is proportional to the intensity of the light focused on the surface of the element. The charge packets from all of the sensors are shifted across the array and read out in the form of electrical information. This electrical information forms an electronic signal which is a representation of the sensed target. From this electronic signal, the location of the target image (or the image of a designated point of the target) on the sensor array can be determined. The image location is indicative of the relative position of the target and the robotic arm.
If the camera is mounted on a common axis with the robotic arm, then precise positioning of the arm can be performed by moving the arm such that the target image impinges on (i.e., is coincident with) the proper pixels of the CCD. However, due to physical constraints, the camera usually cannot be mounted on-axis with the robotic arm. Instead, the camera is mounted off-axis, to one side of the arm. Relocating the camera to this off-axis location results in a phenomenon known as "parallax". "Parallax" is the apparent relocation of an object (e.g., the target) as a result of the relocation of the observer (e.g., the camera).
Parallax can be illustrated by looking at a nearby object with one eye closed and then looking at the same object with the other eye closed. When this is done, the apparent position of the object changes as a result of the different optical path. When the object is closer to the observer, the angle between the two optical paths is greater and hence, the parallax effect is more pronounced.
FIG. 1A illustrates the basic problem of parallax in robotic arm applications. Referring to FIG. 1A, a camera 102 is mounted to a robotic arm 104. Arm 104 is aligned with a worksite 106. Due to parallax, when robot arm 104 is positioned to operate on worksite 106 (i.e. an axis 108 of robot arm 104 is aligned with object 106), worksite 106 appears off-center to camera 102 because it is not on an axis 110 of camera 102. This phenomenon can be illustrated by holding one's finger directly in front of and very close to the left eye, and viewing the finger with the right eye closed. In this situation, the finger is directly centered with the line of vision. However, if the left eye is closed and the right eye opened, the finger appears to the far left of the field of view.
Several techniques have been employed in robotic systems in an attempt to compensate for this problem of parallax. For example, a separate target 112 may be located in a position which is offset to the side of worksite 106. This allows parallax-free positioning provided that the spacing between target 112 and worksite 106 is the same as the spacing between axis 108 and axis 110. However, this required spacing between target 112 and worksite 106 is usually not possible due to size and space limitations in the system. Thus, this technique is normally not practical.
Alternatively, parallax errors may be eliminated by fixing the distance between worksite 106 and target 112 and by fixing the distance between robotic arm 104 and worksite 106. With these distances fixed, the amount of parallax error will be constant and may therefore be corrected. However, if either of the fixed distances varies, then a distance-induced parallax error will occur.
FIG. 1B illustrates a second technique which may be used for parallax correction. In this technique, camera 102 is mounted at a fixed angle .alpha. with respect to robotic arm 104. When a worksite at a point A is located at the intersection of robotic arm axis 128 and camera axis 130, the system is aligned and free of the effects of parallax. However, as with the first conventional technique, this technique is subject to distance-induced parallax errors. In other words, this technique also requires that certain distances be maintained constant. Worksite 106 must be maintained at a fixed distance from robotic arm 104. If the distance varies, distance-induced parallax errors will occur.
Referring again to FIG. 1B, in this configuration, when the system is positioned such that robot arm 104 is aligned with point A at a distance l.sub.0, then camera 102 is also in line with point A. Thus, for a worksite located at a point A which is a fixed distance l.sub.0 from robotic arm 104, the system is aligned and free of the adverse effects of parallax.
However, this configuration is free of parallel errors only for targets at a known and fixed distance l.sub.0. For example, if the target is at point B which is a distance l.sub.1 from arm 104 and camera 102 is aligned with the target, then arm 104 will no longer be aligned with the target. Consequently, robot arm 104 is misaligned due to parallax. The amount of misalignment is a function of the variation in distance.
Note that the system is aligned only when the distance to the target is such that the axis of camera 102 and the axis of robot arm 104 intersect at the target. A similar problem occurs when the target and the worksite to be accessed by the robot are offset.
In many robotic arm applications, target distance cannot be adequately controlled to allow parallex correction in this manner. As a result, distance-induced parallax errors as described above are common. These errors lead to slower positioning, failed positioning, and increased system downtime.
What is needed, then, is a system and method for properly positioning a robotic arm using positioning targets which are located at varying distances from the arm.